Statistical inference in radio astronomy

This thesis unifies several studies, which all are dedicated to the subject of statistical data analysis in radio astronomy and radio astrophysics. Radio astronomy, like astronomy as a whole, has undergone a remarkable development in the past twenty years in introducing new instruments and technologies. New telescopes like the upgraded VLA, LOFAR, or the SKA and its pathfinder missions offer unprecedented sensitivities, previously uncharted frequency domains and unmatched survey capabilities. Many of these have the potential to significantly advance the science of radio astrophysics and cosmology on all scales, from solar and stellar physics, Galactic astrophysics and cosmic magnetic fields, to Galaxy cluster astrophysics and signals from the epoch of reionization. Since then, radio data analysis, calibration and imaging techniques have entered a similar phase of new development to push the boundaries and adapt the field to the new instruments and scientific opportunities. This thesis contributes to these greater developments in two specific subjects, radio interferometric imaging and cosmic magnetic field statistics. Throughout this study, different data analysis techniques are presented and employed in various settings, but all can be summarized under the broad term of statistical infer- ence. This subject encompasses a huge variety of statistical techniques, developed to solve problems in which deductions have to be made from incomplete knowledge, data or measurements. This study focuses especially on Bayesian inference methods that make use of a subjective definition of probabilities, allowing for the expression of probabilities and statistical knowledge prior to an actual measurement. The thesis contains two different sets of application for such techniques. First, situations where a complicated, and generally ill-posed measurement problem can be approached by assuming a statistical signal model prior to infer the desired measured variable. Such a problem very often is met should the measurement device take less data then needed to constrain all degrees of freedom of the problem. The principal case investigated in this thesis is the measurement problem of a radio interferometer, which takes incomplete samples of the Fourier transformed intensity of the radio emission in the sky, such that it is impossible to exactly recover the signal. The new imaging algorithm RESOLVE is presented, optimal for extended radio sources. A first showcase demonstrates the performance of the new technique on real data. Further, a new Bayesian approach to multi-frequency radio interferometric imaging is presented and integrated into RESOLVE. The second field of application are astrophysical problems, in which the inherent stochas- tic nature of a physical process demands a description, where properties of physical quanti- ties can only be statistically estimated. Astrophysical plasmas for instance are very often in a turbulent state, and thus governed by statistical hydrodynamical laws. Two studies are presented that show how properties of turbulent plasma magnetic fields can be inferred from radio observations

Statistical inference in radio astronomy

This thesis unifies several studies, which all are dedicated to the subject of statistical data analysis in radio astronomy and radio astrophysics. Radio astronomy, like astronomy as a whole, has undergone a remarkable development in the past twenty years in introducing new instruments and technologies. New telescopes like the upgraded VLA, LOFAR, or the SKA and its pathfinder missions offer unprecedented sensitivities, previously uncharted frequency domains and unmatched survey capabilities. Many of these have the potential to significantly advance the science of radio astrophysics and cosmology on all scales, from solar and stellar physics, Galactic astrophysics and cosmic magnetic fields, to Galaxy cluster astrophysics and signals from the epoch of reionization. Since then, radio data analysis, calibration and imaging techniques have entered a similar phase of new development to push the boundaries and adapt the field to the new instruments and scientific opportunities. This thesis contributes to these greater developments in two specific subjects, radio interferometric imaging and cosmic magnetic field statistics. Throughout this study, different data analysis techniques are presented and employed in various settings, but all can be summarized under the broad term of statistical infer- ence. This subject encompasses a huge variety of statistical techniques, developed to solve problems in which deductions have to be made from incomplete knowledge, data or measurements. This study focuses especially on Bayesian inference methods that make use of a subjective definition of probabilities, allowing for the expression of probabilities and statistical knowledge prior to an actual measurement. The thesis contains two different sets of application for such techniques. First, situations where a complicated, and generally ill-posed measurement problem can be approached by assuming a statistical signal model prior to infer the desired measured variable. Such a problem very often is met should the measurement device take less data then needed to constrain all degrees of freedom of the problem. The principal case investigated in this thesis is the measurement problem of a radio interferometer, which takes incomplete samples of the Fourier transformed intensity of the radio emission in the sky, such that it is impossible to exactly recover the signal. The new imaging algorithm RESOLVE is presented, optimal for extended radio sources. A first showcase demonstrates the performance of the new technique on real data. Further, a new Bayesian approach to multi-frequency radio interferometric imaging is presented and integrated into RESOLVE. The second field of application are astrophysical problems, in which the inherent stochas- tic nature of a physical process demands a description, where properties of physical quanti- ties can only be statistically estimated. Astrophysical plasmas for instance are very often in a turbulent state, and thus governed by statistical hydrodynamical laws. Two studies are presented that show how properties of turbulent plasma magnetic fields can be inferred from radio observations

Improving self-calibration

Response calibration is the process of inferring how much the measured data depend on the signal one is interested in. It is essential for any quantitative signal estimation on the basis of the data. Here, we investigate self-calibration methods for linear signal measurements and linear dependence of the response on the calibration parameters. The common practice is to augment an external calibration solution using a known reference signal with an internal calibration on the unknown measurement signal itself. Contemporary self-calibration schemes try to find a self-consistent solution for signal and calibration by exploiting redundancies in the measurements. This can be understood in terms of maximizing the joint probability of signal and calibration. However, the full uncertainty structure of this joint probability around its maximum is thereby not taken into account by these schemes. Therefore better schemes -- in sense of minimal square error -- can be designed by accounting for asymmetries in the uncertainty of signal and calibration. We argue that at least a systematic correction of the common self-calibration scheme should be applied in many measurement situations in order to properly treat uncertainties of the signal on which one calibrates. Otherwise the calibration solutions suffer from a systematic bias, which consequently distorts the signal reconstruction. Furthermore, we argue that non-parametric, signal-to-noise filtered calibration should provide more accurate reconstructions than the common bin averages and provide a new, improved self-calibration scheme. We illustrate our findings with a simplistic numerical example.Comment: 17 pages, 3 figures, revised version, title change

Robustness and Explainability of Artificial Intelligence

In the light of the recent advances in artificial intelligence (AI), the serious negative consequences of its use for EU citizens and organisations have led to multiple initiatives from the European Commission to set up the principles of a trustworthy and secure AI. Among the identified requirements, the concepts of robustness and explainability of AI systems have emerged as key elements for a future regulation of this technology. This Technical Report by the European Commission Joint Research Centre (JRC) aims to contribute to this movement for the establishment of a sound regulatory framework for AI, by making the connection between the principles embodied in current regulations regarding to the cybersecurity of digital systems and the protection of data, the policy activities concerning AI, and the technical discussions within the scientific community of AI, in particular in the field of machine learning, that is largely at the origin of the recent advancements of this technology. The individual objectives of this report are to provide a policy-oriented description of the current perspectives of AI and its implications in society, an objective view on the current landscape of AI, focusing of the aspects of robustness and explainability. This also include a technical discussion of the current risks associated with AI in terms of security, safety, and data protection, and a presentation of the scientific solutions that are currently under active development in the AI community to mitigate these risks. This report puts forward several policy-related considerations for the attention of policy makers to establish a set of standardisation and certification tools for AI. First, the development of methodologies to evaluate the impacts of AI on society, built on the model of the Data Protection Impact Assessments (DPIA) introduced in the General Data Protection Regulation (GDPR), is discussed. Secondly, a focus is made on the establishment of methodologies to assess the robustness of systems that would be adapted to the context of use. This would come along with the identification of known vulnerabilities of AI systems, and the technical solutions that have been proposed in the scientific community to address them. Finally, the promotion of transparency systems in sensitive systems is discussed, through the implementation of explainability-by-design approaches in AI components that would provide guarantee of the respect of the fundamental rights.JRC.E.3-Cyber and Digital Citizens' Securit

Statistical properties of Faraday rotation measure from large-scale magnetic fields in intervening disc galaxies

To constrain the large-scale magnetic field strengths in cosmologically distant galax- ies, we derive the probability distribution function of Faraday rotation measure (RM) when random lines of sight pass through a sample of disc galaxies, with axisymmetric large-scale magnetic fields. We find that the width of the RM distribution of the galaxy sample is directly related to the mean large-scale field strength of the galaxy population, provided the dispersion within the sample is lower than the mean value. In the absence of additional constraints on parameters describing the magneto-ionic medium of the intervening galaxies, and in the situation where RMs produced in the intervening galaxies have already been statistically isolated from other RM contributions along the lines of sight, our simple model of the magneto-ionic medium in disc galaxies suggests that the mean large-scale magnetic field of the population can be measured to within ~ 50% accuracy.Comment: 4 pages, Proceedings of FM8 "New Insights in Extragalactic Magnetic Fields", XXXth General Assembly of the IAU, Vienna, August 20-31, 201

Estimating extragalactic Faraday rotation

(abridged) Observations of Faraday rotation for extragalactic sources probe magnetic fields both inside and outside the Milky Way. Building on our earlier estimate of the Galactic contribution, we set out to estimate the extragalactic contributions. We discuss the problems involved; in particular, we point out that taking the difference between the observed values and the Galactic foreground reconstruction is not a good estimate for the extragalactic contributions. We point out a degeneracy between the contributions to the observed values due to extragalactic magnetic fields and observational noise and comment on the dangers of over-interpreting an estimate without taking into account its uncertainty information. To overcome these difficulties, we develop an extended reconstruction algorithm based on the assumption that the observational uncertainties are accurately described for a subset of the data, which can overcome the degeneracy with the extragalactic contributions. We present a probabilistic derivation of the algorithm and demonstrate its performance using a simulation, yielding a high quality reconstruction of the Galactic Faraday rotation foreground, a precise estimate of the typical extragalactic contribution, and a well-defined probabilistic description of the extragalactic contribution for each data point. We then apply this reconstruction technique to a catalog of Faraday rotation observations. We vary our assumptions about the data, showing that the dispersion of extragalactic contributions to observed Faraday depths is most likely lower than 7 rad/m^2, in agreement with earlier results, and that the extragalactic contribution to an individual data point is poorly constrained by the data in most cases.Comment: 20 + 6 pages, 19 figures; minor changes after bug-fix; version accepted for publication by A&A; results are available at http://www.mpa-garching.mpg.de/ift/faraday

Extragalactic magnetic fields

Invited Poster presentato in remoto alla conferenza "The First Pietro Baracchi Conference: Italo-Australian Radio Astronomy in the Era of the SKA", 1-4 November 2016 — ARRC Building, Perth, Western Australi

Statistical methods for the analysis of rotation measure grids in large scale structures in the SKA era

To better understand the origin and properties of cosmological magnetic fields, a detailed knowledge of magnetic fields in the large-scale structure of the Universe (galaxy clusters, filaments) is crucial. We propose a new statistical approach to study magnetic fields on large scales with the rotation measure grid data that will be obtained with the new generation of radio interferometers.Comment: 9 pages; to appear as part of 'Cosmic Magnetism' in Proceedings 'Advancing Astrophysics with the SKA (AASKA14)', PoS(AASKA14)11

Artificial Intelligence and Digital Transformation: early lessons from the COVID-19 crisis

The COVID-19 pandemic has created an extraordinary medical, economic and social emergency. To contain the spread of the virus, many countries adopted a lock down policy closing schools and business and keeping people at home for several months. This resulted in a massive surge of activity online for education, business, public administration, research, social interaction. This report considers these recent developments and identifies some early lessons with respect to the present and future development of AI and digital transformation in Europe, focusing in particular on data, as this is an area of significant shifts in attitudes and policy. The report analyses the increasing use of AI in medicine and healthcare, the tensions in data sharing between individual rights and collective wellbeing, the search for technological solutions like contact tracing apps to help monitor the spread of the virus, and the potential concerns they raise. The forced transition to online showed the resilience of the Internet but also the disproportionate impact on already vulnerable groups like the elderly and children. The report concludes that the COVID-19 crisis has acted as a boost for AI adoption and data sharing, and created new opportunities. It has also amplified concerns for democracy and social inequality and showed Europe’s vulnerability on data and platforms, calling for action to address these crucial aspects.JRC.B.6-Digital Econom